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ABSTRACT 



This is one of a series of reports that document the 



formative research that supported the development of the "First Steps" 
program designed to improve the literacy and numeracy of primary school 
students in Western Australia, particularly those considered at-risk. The 
four themes of First Steps (reading, writing, spelling, and oral language) 
are organized around a development continuum of milestones along the child's 
path to literacy. This interim report presents preliminary findings from two 
tests administered to First Steps students as the first part of a time series 
study. The main aim of this report is to examine the impact of First Steps on 
the reading and writing ability of Year 5 students while taking into account 
the effect of socioeconomic status background. Reading comprehension data 
were available from 665 year-5 students in 31 classes in 19 schools, and 
writing ability data came from 736 students. There was evidence that First 
Steps is making a difference in the reading ability of students, with 
suggestions that the effect of First Steps seems stronger for children of low 
socioeconomic status background. Results from the writing exercise suggest 
that First Steps is having a positive effect on the writing ability of 
children regardless of their socioeconomic status. Eight appendixes present 
statistical information, details about the reading and writing tests, and 
further descriptions of analyses procedures. (Contains 19 exhibits and 3 
references.) (SLD) 
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Editor’s Note 



This document is one of a series of reports that document the formative research that supported 
the creation and development of First Steps™. As a result of this research, the Education 
Department of Western Australia (EDWA), in collaboration with the Australian Council for 
Educational Research (ACER) revised First Steps in response to each of the issues and questions 
raised by this research. First Steps training courses, Developmental Continua, and Resource 
Books are published with due amendments and alterations. 

Other research documents that support the development of First Steps include: 

Dr. Phil Deschamp: 

♦ A Survey of the Implementation of the Literacy Component of the First Steps Project in WA 

♦ The Implementation of The Literacy Component of The First Steps Project in ELAN Schools 

♦ A Survey of the Effectiveness of the Focus Teacher ‘B’ Training for the First Steps Project 

♦ Student Achievement: A Study of the Effects of First Steps Teaching on Student 
Achievement 

♦ Case Studies of The Implementation of the First Steps Project in Twelve Schools 

♦ The Development and Implementation of the First Steps Project in Western Australia 



ACER: 

♦ Empirical Validation of the First Steps Reading Continuum 

♦ Empirical Validation of the First Steps Spelling and Writing Continua 

♦ Empirical Re-Validation of the First Steps Spelling Continuum 

♦ Assessment and Record of the Changes made to the Spelling Continuum 

♦ The Impact of First Steps on Schools and Teachers 

♦ The Impact of First Steps on the Reading and Writing Ability of Year 5 Students 

♦ Background: First Steps and the ACER Evaluation 8c Report on the Validity of the First 
Steps Writing and Spelling Continua* 



EDWA: 

♦ Supporting Linguistic and Cultural Diversity Through First Steps: The Highgate Project 



For more information about on-going First Steps research, please contact: 

First Steps™ / Heinemann 
361 Hanover Street 
Portsmouth, NH 03801-3912 
1.800.541.2086, ext. 281 
firststeps@heienmann.com 
www.heinemann.com/firststeps 



THE IMPACT OF FIRST STEPS ON THE READING AND WRITING 
ABILITY OF WESTERN AUSTRALIAN YEAR 5 SCHOOL STUDENTS 



an interim report to the 
Curriculum Development Branch 
Western Australian Ministry of Education 



Australian Council for Educational Research, May 1993. 



This is the second of three reports on the ACER 1992 evaluation of the First Steps project in 
Western Australia. The first report describes the empirical validation of the First Steps Spelling 
and Writing continua. The third report examines the responses of teachers to various aspects of 
the First Steps project. Hueli of these reports is available from the Curriculum Development 
Branch of the Ministry of Education. 



Thanks to all the schools who agreed to participate in the evaluation of First Steps, and thanks 
to all the students, teachers, principals and administrative staff in schools who so generously 
contributed towards it. In particular thanks to the students who completed the writing and 
reading tests and the teachers who administered and commented upon them. The results from 
these tests are used in this report. 



SUMMARY OF THE MAIN FINDINGS 



The following conclusions can be drawn from the data analysed in this report: 

* If students from old First Steps schools had lower average levels of reading 
comprehension ability than students in new or non First Steps schools before First Steps 
was introduced, then it is likely that First Steps has lead to an improvement in the 
reading ability of those students exposed to the program. 

* If students from old First Steps schools had lower average levels of writing ability 
than students in new or non First Steps schools before First Steps was introduced, then 
it is likely that First Steps has lead to an improvement in the writing ability of those 
students exposed to the program. 

* There is some evidence that First Steps may be making an important difference in the 
reading ability of students. Some informed guessing of likely mean scores before the 
introduction of First Steps suggests that the mean TORCH reading score of children in 
old First Steps schools may have moved from ‘the low’ range to within in the ‘average’ 
range of scores. This improvement seems to be due to the effects of First Steps. 
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APPENDICES 



Appendix A. 


Weighting of the sample of Year 5 students. Appendix A is printed on pink 
paper. 


Appendix B. 


Texts and answer sheets used with the TORCH reading comprehension test. 
Instructions to the teachers are also attached. Appendix B is printed on blue 
paper. 


Appendix C. 


Graphical display showing the estimated range of difficulty for each of the texts 
used with the TORCH reading comprehension test. Appendix C is printed on 
dark green paper. 


Appendix D. 


Text of the forms used for the Writing exercise, including instructions to the 
teachers. Appendix D is printed on light green paper. 


Appendix E. 


The ACER Criteria for the Assessment of Writing. Appendix E is printed on 
buff paper. 


Appendix F. 


Percentile ranks and stanines compared with standard deviations under the 
normal curve. Appendix F is printed on orange paper. 


Appendix G. 


The reason for not using tests of statistical significance. Appendix G. is printed 
on dark blue paper 


Appendix H. 


A description of the ANOVA procedure. Appendix H. is printed on dark pink 



INTRODUCTION 



Background to the First Steps Project 

First Steps is a program instituted by the WA Ministry of Education to improve the literacy and 
numeracy of primary school students. It is intended, in particular, to assist in the development 
of the literacy and numeracy skills of ‘at risk' students. First Steps was introduced in 1988 and 
- has been evolving since that time. So far most of the work produced as part of the First Steps 
project has focussed upon literacy. 

Background to the evaluation of First Steps 

The WA Ministry of Education approached ACER in early 1992 to evaluate First Steps. Work 
for the evaluation began in April 1992. The evaluation was designed to measure the effects of 
First Steps at three levels. These levels were: (1) the school (2) the teachers and (3) the 
students. Questionnaires were used to gather data to measure the effects of First Steps at the 
school and teacher level. Reading and writing tests were used to try and establish the effects of 
First Steps at the student level. It is the data taken from these tests that are reported here. The 
evaluation was also designed to validate the First Steps continua. (See The Impact of First 
Steps on Schools and Teachers and Empirical Validation of the First Steps Spelling and Writing 
Continua. *) 

Aims of this report 

This is an interim report. It aims to provide preliminary findings derived from two tests that 
were administered as the first part of a time series study. This study aims to examine the 
impact of First Steps on the reading and writing abilities of children. Originally, it was planned 
to administer one round of tests each year for three years. At the time of this report being 
prepared, however, it is unclear whether the next two rounds of tests will be administered. 
Consequently, the data presented here are explored on the assumption that these are likely to be 
the only test data measuring reading and writing ability of children available lor this evaluation 
of First Steps. Within these limitations, the main aim of this report is to examine what impact 
First Steps has on the reading and writing ability of Year 5 students while taking into account 
the effect of socio-economic status background. 



1 These reports are part of the series of three reports produced as part of the ACER 1992 evaluation of 
First Steps. 
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The sample of students 



The sample was stratified into six groups based on the level of disadvantage of the schools and 
the level of involvement of the school in First Steps. Exhibit 1 shows how the six strata were 
derived. 



Exhibit 1 : Strata used in drawing the sample of Year 5 students for the administration of reading 
and writing tests. 



First Steps status of school 



Disadvantage status 
of school 


1 Non First Steps 


New First Steps 


Old First Steps 


disadvantaged (PSP) 


| Non FS PSP 


New FS PSP 


Old FS PSP 


Non Disadvantaged 


| Non FS non PSP 


New FS non PSP 


Old FS non PSP 



A non First Steps school was defined as one that had had no formal involvement with First 
Steps. It should be noted, however, that within these non-First Steps schools there were many 
teachers who used First Steps materials and ideas. ^ A new First Steps school was defined as 
one with less than 12 months formal involvement with First Steps. The level of disadvantage of 
a school was defined using the Ministry of Education’s formal definition of disadvantage - its 
‘PSP’ status. 

The schools in each stratum were selected with a probability proportional to size. In other 
words, the more students there were in a school, the more likely the school was to be chosen. If 
school size is not taken into account when sampling then small schools tend to be over 
represented in the sample. Four schools were selected from each stratum, so a total of 24 
schools was chosen. 



Unless there were more than two Year 5 classes in a school all Year 5 students within a school 
were included in the sample. Where there were more than two Year 5 classes, two classes were 
randomly selected and only students from these two classes were chosen. A Year 5 class was 
defined as any class in which there was at least one Year 5 student. Thus, Year 5 students from 
composite classes were sometimes selected. 

Not all selected schools or classes within schools chose to participate in the evaluation. The 
final set of reading comprehension data consisted of 665 valid responses from 31 classes in 19 
schools. The writing ability data consisted of 736 valid responses from 33 classes in 19 




Responses to questionnaires sent to teachers within these schools showed that nearly two thirds of 
Year 1 and Year 5 teachers used some First Steps ideas. Some had recently come from schools that had 
implemented First Steps and others had obtained First Steps materials from colleagues and used these 
materials without having had any forma] involvement with First Steps. 
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